STA6235: Modeling in Regression
\ln \left( \frac{\pi}{1-\pi} \right) = \beta_0 + \beta_1 x_1 + ... + \beta_k x_k,
where \pi = \text{P}[Y = 1] = the probability of the outcome/event.
We also discussed that we provide interpretations for e^{\hat{\beta}_i}, the odds ratio for slope i.
Suppose our response variable now has c ordered categories
We now will expand and learn ordinal logistic regression
Under ordinal logistic regression, we will create c-1 models.
The \hat{\beta}_i will be the same across the models.
The \hat{\beta}_0 will change for each category.
Using the cumulative logit model,
\begin{align*} \text{logit}\left( P[Y \le j] \right) &= \hat{\beta}_{0j} + \hat{\beta}_{1} x_1 +... + \hat{\beta}_{k} x_k \\ \log \left( \frac{P[Y \le j]}{1 - P[Y \le j]} \right)&= \hat{\beta}_{0j} + \hat{\beta}_{1} x_1 + ... + \hat{\beta}_{k} x_k \\ \log \left( \frac{\pi_1 + ... + \pi_j }{\pi_{j+1} + ... + \pi_{c}} \right) &= \hat{\beta}_{0j} + \hat{\beta}_{1} x_1 + ... + \hat{\beta}_{k} x_k \end{align*}
\log \left( \frac{\pi_1 + ... + \pi_j }{\pi_{j+1} + ... + \pi_{c}} \right) = \hat{\beta}_{0j} + \hat{\beta}_{1} X_1 + ... + \hat{\beta}_{k} X_k
As noted previously, the intercept depends on j.
This means that curves will have the same shape \forall \ j.
We are just shifting the curve along the x-axis, depending on the response category.
This model assumes proportional odds.
We will use the polr() function from the MASS package.
MASS after you have loaded tidyverse, it will overwrite the select() function from tidyverse. To avoid this, we will call the function using MASS::polr().lm() and glm() functions, we will run model results through the summarize() function.Let’s consider data from a General Social Survey, relating political ideology to political party affiliation. Political ideology has a five-point ordinal scale, ranging from very liberal (Y=1) to very conservative (Y=5). Let x be an indicator variable for political party, with x = 1 for Democrats and x = 0 for Republicans. We will construct an ordinal logistic regression model that models political ideology as a function of political party and sex.
Call:
MASS::polr(formula = Ideology ~ Party + Sex, data = gss, Hess = TRUE)
Coefficients:
Value Std. Error t value
PartyRepublican 0.9636 0.1297 7.4311
SexMale 0.1169 0.1273 0.9177
Intercepts:
Value Std. Error t value
1 - Very Liberal|2 - Liberal -1.4518 0.1226 -11.8373
2 - Liberal|3 - Moderate -0.4583 0.1048 -4.3746
3 - Moderate|4 - Conservative 1.2550 0.1142 10.9873
4 - Conservative|5 - Very Conservative 2.0890 0.1293 16.1587
Residual Deviance: 2474.142
AIC: 2486.142 \begin{align*} \text{logit}\left( P[Y \le \text{V. Lib.}] \right) &= -1.45 + 0.96 \text{repub.} + 0.12 \text{male} \\ \text{logit}\left( P[Y \le \text{Lib.}] \right) &= -0.46 + 0.96 \text{repub.} + 0.12 \text{male} \\ \text{logit}\left( P[Y \le \text{Mod.}] \right) &= 1.26 + 0.96 \text{repub.} +0.12 \text{male} \\ \text{logit}\left( P[Y \le \text{Cons.}] \right) &= 2.09 + 0.96 \text{repub.} +0.12 \text{male} \end{align*}
Odds ratios are interpreted slightly different due to the model being cumulative.
For continuous predictors:
For a one [predictor unit] increase in [predictor], the odds in favor of [the response category j] or lower, as compared to higher than [the response category j], are multiplied by e^{\hat{\beta}_i}.
For a one [predictor unit] increase in [predictor], the odds in favor of [the response category j] or lower, as compared to higher than [the response category j], are [increased or decreased] by [100(e^{\hat{\beta}_i}-1)% or 100(1-e^{\hat{\beta}_i})%].
Odds ratios are interpreted slightly different due to the model being cumulative.
For categorical predictors:
As compared to [the reference category], the odds in favor of [the response category j] or lower, as compared to higher than [the response category j], for [the predictor category of interest] are multiplied by e^{\hat{\beta}_i}.
As compared to [the reference category], the odds in favor of [the response category j] or lower, as compared to higher than [the response category j], for [the predictor category of interest] are [increased or decreased] by [100(e^{\hat{\beta}_i}-1)% or 100(1-e^{\hat{\beta}_i})%].
For any fixed response, the estimated odds that a Republican’s response is in the conservative direction rather than the liberal direction are e^{0.9636} = 2.62 times the estimated odds for Democrats.
For any fixed response, the estimated odds that a male’s response is in the conservative direction rather than the liberal direction are e^{0.1169} = 1.12 times the estimated odds for females.
Because summary() does not return p-values, we will use the full/reduced approach, removing a single predictor at a time.
confint() function.The 95% CI for the OR for
party affiliation (republican vs. democrat) is (2.04, 3.38),
and for biological sex (male vs. female) is (0.88, 1.44).
As mentioned previously, we are assuming proportional odds.
We will check this assumption with Brant’s test ((https://www.jstor.org/stable/2532457)[article here]).
Briefly, this will construct a \chi^2 test for every predictor in the model.
If p<\alpha, the assumption is broken.
If the assumption is broken, we should step down to nominal logistic regression.
brant() function from the brant package.----------------------------------------------------
Test for X2 df probability
----------------------------------------------------
Omnibus 11.88 6 0.06
PartyRepublican 3.68 3 0.3
SexMale 8.04 3 0.05
----------------------------------------------------
H0: Parallel Regression Assumption holds